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Probabilistic  Analysis  of  Time  Sensitive  Systems 


Problem  Statement 


Time-sensitive  systems  in  uncertain  environments  have  complex 
behaviors.  How  do  we  assure  correctness  of  such  systems? 

-  Exact  probabilistic  verification  is  infeasible  due  to  model  size 

-  Black  box  testing  does  not  yield  bounded  predictions 

-  Need  formal  approach  for  dealing  with  uncertainty 

-  Accurate,  bounded,  probabilistic  results 

-  In  reasonable  time  even  for  rarely  occurring  errors 


Stochastic  Model  Checking  (SMC) 


SMC  is  a  rigorous  simulation-based  approach  for  estimating  that  a 
property  holds  in  a  system. 

-  System  properties  described  in  formal  language  (BLTL,  etc.) 

-  Property  is  tested  on  “sample  trajectories”  (sequence  of  states) 

-  Each  outcome  treated  as  a  Bernoulli  trial  (i.e.,  coin  flip) 


Semantic  Importance  Sampling 

A  New  Approach  to  Importance  Sampling 


Input  Specification  in  C 


SMT2  Model 


#include  "osmosis_client . h" 

//@dist  a=unif orm (min=0 , max=5 ) 

/ /@dist  b=normal (mean=3 , std=l , min=0 , max=5 ) 
void  simple ( ) 

{ 

double  a  =  INPUT_D ( "a" ) ; 
double  b  =  INPUT_D ( "b" ) ; 
double  c  =  a  +  b; 
double  d  =  (a  -  b)/2.0; 

ASSERT  ( sin  (c) ’^cos  (d/2  )  <  0.995); 


Translate  C 
model  to  SMT2 
for  Analysis. 


Abstract  Indicator 
Function  I*(x) 


Recursively  invoke 
dReal  SMT  checker  to 
build  abstract  model 
of  specification. 
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SMC  Basics 

•  Indicator  function  I(x)  =  1  iff  property  holds  for  input  x. 

/  vcltC'd^ 

•  Relative  Error  REi^p)  =  ^  ^  ^  is  measure  of  accuracy. 

•  Draw  random  samples  from  input  distribution  /(x)  until 
target  Relative  Error  is  met. 

•  Estimated  probability  that  property  holds  is: 
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(set-logic  QF_NRA) 

(declare-fun  a  ()  Real) 

(declare-fun  b  ()  Real) 

(declare-fun  a_l  ()  Real) 

(declare-fun  b_l  ()  Real) 

(declare-fun  c_l  ()  Real) 

(declare-fun  d_l  ()  Real) 

(assert  (>=  a  0) ) 

(assert  (<=  a  5) ) 

(assert  (>=  b  0) ) 

(assert  (<=  b  5) ) 

(assert  (=  a_l  a) ) 

(assert  (=  b_l  b) ) 

(assert  (=  c_l  (+  a_l  b_l) ) ) 

(assert  (=  d_l  (/  (-  a_l  b_l)  2.0))) 

(assert  (not  (<  (*  (sin  cl)  (cos  d  1) )  0.9))) 
(check-sat) 

(exit) 
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ASSERTO 


Input  Generation 
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Use  I*(x)  to  generate  random 
input  vectors: 

•  Randomly  pick  SAT  cube 

•  Randomly  pick  point  in  cube 
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Apply  inverse  CDF  on 
each  input  variable. 
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Weight  function  W(Xi) 
is  probability  p*  that  x 
is  in  /*(x). 

Abstract  Probability 

Number  of 
5  cubes  in  r(x). 


Raw  Prob.  Estimate 
Pvaw  O'  024 


ia,b)  =  (2.115,2.503) 

Apply  generated  inputs  to 
original  C  model  to  calculate 
bounded  failure  probability 
estimate. 
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Level  of  cubes 


Importance  Sampling 


Modify  input  distribution  to  make  rare  properties  more  visible. 
Weighting  function  W{x)  maps  solution  back  to  original  problem. 
Reduced  relative  error  with  same  number  of  samples. 
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Final  Probability  Estimate 
P  =  P*Praw  =  0-  00047 

RE(p)  =  0. 01 
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